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Abstract Achieving sustained student engagement with practice in computer-based 
writing strategy training can be a challenge. One potential solution is to foster engagement 
by embedding practice in educational games; yet there is currently little research com- 
paring the effectiveness of game-based practice versus more traditional forms of practice. 
In this study, the ARCS model (Keller, Perform Instr 26(8):1—7, 1987b) was used to 
investigate the motivational characteristics of different practice conditions. To this end, 
175 students were randomly assigned to one of four experimental conditions: game-based, 
question-based, model-based, and writing-based practice. All students first learned strat- 
egies to write an essay introduction. Subsequently, students practiced using the strategies 
in the four different conditions. Game-based practice was expected to positively affect 
ARCS-related motivation toward practice. Results showed that students perceived game- 
based practice as significantly more interesting and engaging than question-based practice. 
However, although game-based practice was perceived more positively, only model-based 
and question-based practice demonstrated a beneficial impact on students’ ability to 
implement the writing strategies. These results underline the necessity of interconnecting 
motivational and instructional design when developing practice methods for computer- 
based writing strategy training. 


Keywords Serious games - Game-based learning - Writing strategy instruction - ARCS 
model 


A. Proske (DX) 
Psychology of Learning and Instruction, TU Dresden, Zellescher Weg 17, 01062 Dresden, Germany 
e-mail: antje.proske @tu-dresden.de 


R. D. Roscoe 
Human Systems Engineering, Arizona State University, 7271 E. Sonoran Arroyo Mall, Santa Catalina 
150D, Mesa, AZ 85212, USA 


D. S. McNamara 


Psychology Department, Learning Sciences Institute, Arizona State University, 
P.O. Box 872111, Tempe, AZ 85287-2111, USA 


va Springer 


482 A. Proske et al. 


Introduction 


Writing is essential to success, yet many students lack writing expertise and fail to master 
basic writing skills during their formative education years. One means of developing 
writing skills is through instruction in writing strategies (e.g., De La Paz and Graham 2002; 
Graham 2006; Graham and Perin 2007; Kellogg and Whiteford 2009). Writing strategy 
instruction explicitly and systematically teaches students to use strategies for planning, 
drafting, and revising (e.g., Graham and Perin 2007), which entails explaining and mod- 
eling each strategy and providing students with ample opportunities for sustained practice 
(Roscoe et al. 2013). Research has demonstrated that such instruction is beneficial to 
students, particularly less skilled writers (e.g., Graham 2006). 

Few studies have contrasted the relative merits of different forms of practice in writing 
strategy instruction (Zimmerman and Kitsantas 2002), and research has specifically 
neglected how these forms of practice engage (or fail to engage) students motivationally. In 
this case, we use the term motivation to refer to individuals’ intrinsic and extrinsic goals to 
achieve or avoid a given outcome, which in turn influences their choices and expenditure 
of effort (e.g., Keller 1999). Across different modes of practice, motivation may play a 
crucial role in students’ willingness to practice and perhaps place an upper bound on the 
effectiveness of writing strategy instruction (Kellogg and Whiteford 2009, p. 254). As 
such, a deeper understanding of students’ motivation related to the instructional design of 
practice opportunities (Keller 1987b) may help us elicit greater student investment in 
strategy practice (Huang et al. 2006). 

In this study, we consider design in motivating practice opportunities within computer- 
based writing strategy training. Specifically, we contrast the benefits of game-based and 
more conventional forms of practice on students’ perceived motivation (as defined by the 
ARCS motivational design model; Keller 1987b) and students’ learning and application of 
writing strategies. 


Applying the ARCS motivational design model to writing strategy practice 


The ARCS model posits four conceptual components (i.e. Attention, Relevance, Confi- 
dence, and Satisfaction) that must be fulfilled for people to become and remain motivated 
(Keller 1987b). Attention refers to learners’ responses to the provided instructional 
materials and serves as a prerequisite for learning. Attention-building principles arouse and 
sustain curiosity and interest in the instructional content. After obtaining learners’ interest, 
perceived Relevance of instruction plays an important role (e.g., Hidi et al. 2002). Rele- 
vance-building principles link instructional materials to students’ prior learning experi- 
ences, learning objectives, and the future use of acquired skills. Motivation is further 
influenced by learners’ beliefs or Confidence that success is possible or probable (e.g., 
Cheng and Yeh 2009; Pajares 2003). Confidence-building principles foster expectations of 
success and encourage students to perceive success as under their control. Finally, practice, 
feedback, and positive attitudes towards the learning content promote Satisfaction, which 
positively affects students’ further motivation (e.g., Keller 1999). Empirical findings show 
a positive relationship between motivational processing (Attention, Relevance, and Con- 
fidence) and Satisfaction (e.g., Huang et al. 2010). Moreover, research suggests that 
incorporating ARCS design principles enhances the motivational impact of instructional 
materials and activities (e.g., Cheng and Yeh 2009; Huang et al. 2006; Jakobsdottir and 
Hooper 1995; Keller 1999). 
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Conventional forms of writing strategy practice task students with studying and imi- 
tating model texts and/or applying novel strategies in their own writing (e.g., Graham and 
Perin 2007). Both common forms of strategy practice can enhance students’ writing skills 
compared to general instructions for simply working productively, making the paper better, 
and writing a paragraph (e.g., Graham et al. 1995; Zimmerman and Kitsantas 1999), 
freewriting without explicit instruction, or questions guiding the writing (e.g., Knudson 
1989). These conventional forms of practice can also align with ARCS principles. Studying 
and imitating model texts is a form of practice that engages student reflection, which is 
related to Keller’s (1987b) attention-building principle of inquiry arousal. Model texts also 
demonstrate what students can and should strive for in their own writing (e.g., Knudson 
1989). Thus, when reading model texts, students are provided with authentic and useful 
examples that emphasize the relevance of the practice. In contrast, strategy application 
practice can engage relevance-building principles such as using goal-oriented tasks and 
matching students’ motives (Keller 1987b) because it allows students to learn by doing 
(e.g., Schunk and Swartz 1993; Zimmerman and Kitsantas 1999). To successfully apply the 
strategy, the student must mobilize and sustain effort which, particularly when combined 
with feedback, can also improve students’ self-efficacy (confidence) for writing (e.g., 
Schunk and Swartz 1993). 

As an alternative to more conventional approaches, practice tasks can also be embedded 
within games or game-like environments. Game-based practice is increasingly recognized 
as a means to support not only skill acquisition, but also knowledge acquisition and 
strategy automaticity (e.g., Van Eck 2006). Games are interactive, goal-directed problem- 
solving activities guided by rules (e.g., Wouters et al. 2013). As students play and strive to 
overcome game challenges, they must gradually practice and master the knowledge, 
strategies, and skills required to win. In addition, games also provide feedback that allows 
players to monitor their progress (e.g., Dempsey et al. 2002; Wouters et al. 2013), and this 
feedback can serve an important prompt for self-regulated learning and growth (Butler and 
Winne 1995; Narciss 2008; Shute 2008). 

In educational games, the objective is to leverage core game features and game 
interactions in ways that support more formal learning (e.g., Roscoe et al. 2013). Several 
taxonomies describe potentially engaging game features, such as fantasy, narrative, sen- 
sory stimuli, rules, goals, challenge, assessment, competition, and control (e.g., Bedwell 
et al. 2012; Garris et al. 2002; Wilson et al. 2009). Games often incorporate elements of 
fantasy or narrative that contextualize the interaction and provide a sense of purpose to the 
tasks. Narratives generally consist of characters, protagonists, plots, conflict, and resolu- 
tion. Combining these elements can increase uncertainty and build suspense, and also 
increase a sense of immersion, presence, and flow (Barab et al. 2010). Likewise, games 
with various sensory stimuli, such as moving targets, can help to sustain attention 
(Dempsey et al. 2002). A specific set of rules helps learners focus on what to do in order to 
attain the goal of the game (e.g., Dempsey et al. 2002). Thus, rules support learners’ active 
participation in the game. Furthermore, the goal structure of the game is made transparent 
and clear to the learner which may foster performance (e.g., Garris et al. 2002; Latham and 
Locke 2007). Challenge, assessment, competition, and control may help students to judge 
their learning progress (e.g., Garris et al. 2002). Together, such game features are assumed 
to enhance students’ enjoyment and engagement in playing the game which in turn may 
increase students interest in continuing to practice the strategies (e.g., Roscoe et al. 2013). 

Within educational gaming, mini-games represent one common approach in which 
simple games focus on a concrete skill or topic (Panagiotakopoulos 2011; Smith and 
Sanchez 2010). Mini-games often employ a question—answer approach similar to testing 
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during strategy training (Ricci et al. 1996). Such an approach is similar to taking a test on 
learning material, which has been shown to improve retention compared to spending an 
equivalent amount of time restudying the material (e.g., Roediger and Karpicke 2006). 
Benefits from testing are generally attributed to the retrieval processes in which students 
engage during testing, such as elaboration or creating multiple retrieval routes. Interactive, 
computer-based questions have the additional potential of providing immediate feedback 
to the student (Proske et al. 2012). Interactive question answering requires ongoing 
interaction between the student and the computer in which students solve the presented 
question. This fulfills Keller’s (1987b) attention-building principle of stimulating an atti- 
tude of inquiry. Furthermore, giving students feedback and opportunities for repetition and 
correction potentially scaffolds them toward independently correcting incorrect solutions 
(McDaniel and Wooldridge 2012). In this way, confidence-building principles are 
implemented. 

From an instructional point of view, educational mini-games employing a question— 
answer approach offer a number of promising motivational attributes for practicing and 
applying writing strategies. For example, students might build confidence for the strategies 
by practicing them in the games before incorporating these strategies in their own essays 
(Roscoe et al. 2013). Combinations of graphics, sound, and narrative could be used to 
enhance the sensory experience, fulfilling Keller’s attention-building principle for arousing 
curiosity and interest (e.g., Gunter et al. 2008; Rieber 1996). Mini-games also have clear 
goals, simple rules, and are relatively easy to play (e.g., Panagiotakopoulos 2011; Paras 
and Bizzocchi 2005), which allows students to repeatedly focus on applying writing 
strategies with a goal of learning and improving (Kellogg and Whiteford 2009). This focus 
aligns with Keller’s principle of building relevance via goal orientation and motive 
matching. As with any game, mini-games can provide timely information to players about 
their performance (i.e. feedback, Roscoe et al. 2013). In this way, students’ confidence and 
satisfaction can be fostered (e.g., Paras and Bizzocchi 2005). In sum, practicing by means 
of mini-games can fulfill a majority of the motivational design principles described by 
Keller (1987b). In practical terms, mini-games are also relatively easy and cheap to 
develop and integrate within computer-based training compared to complex games with 
lengthy narratives, intricate game mechanics, and high-end graphics. 

Empirical studies have examined collections of mini-games focused on specific tasks or 
skills, including strategy games that target problem-solving and decision-making (e.g., Ke 
2008) and adventure games focused on reading and math skills (e.g., Rosas et al. 2003). 
Students have typically played these games over several weeks for at least 1 h per week. 
During this time, control groups either remained in their teacher-led classrooms (Rosas 
et al. 2003) or completed paper-and-pencil drills similar to the mini-games (Ke 2008). 
Such research has consistently reported positive effects on motivation but also a lack of 
performance and achievement differences between conventional practice and game-based 
practice (e.g., Ke 2008; Rosas et al. 2003). 

Few researchers have investigated the effectiveness of mini-games under shorter 
practice durations (<2 h), but such studies also suggest that mini-games may increase 
students’ motivation to practice (e.g., Jackson et al. 2011; Klein and Freitag 1991). Mixed 
results have been reported for achievement. Mini-games have been found to be equivalent 
to worksheets including the same practice questions (Klein and Freitag 1991) or associated 
with lower achievement compared to guided practice with feedback (e.g., Jackson et al. 
2011). In the latter study, however, only the traditional practice group was allowed to 
improve their initial answer after receiving feedback. Students in the game-based condition 
received feedback on the quality of their answer but were not allowed to revise. Thus, 
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differences in the instructional design may account for decrements in performance with 
game-based practice. 


The present study 


One purpose of this study is to further investigate the extent to which students perceive 
game-based practice as more motivating, according to the ARCS motivational design 
model, compared to more traditional methods of practice in computer-based writing 
strategy training. ARCS-related motivation is a situational state that affects the level of 
concentration and attention toward an assigned learning task (Rodgers and Withrow- 
Thorton 2005). Notably, we do not, in this study, investigate motivational gains by 
comparing pretest and posttest motivation to learn. 

Table | compares the presence of the ARCS design principles in game-based practice to 
two conventional forms of practice (model-based practice and writing-based practice). To 
equate the materials used within the mini-game, Table | also compares game-based 
practice to a similar question—answer condition without game attributes. We hypothesize 
that mini-games for writing strategy practice will positively influence students’ reported 
attention, confidence, and satisfaction compared to conventional forms of writing strategy 
practice (i.e. studying and imitating model texts and applying novel strategies in one’s own 
writing). We further expect game-based practice to grab more attention of students than a 
similar non-game question—answer condition. No differences were expected between the 
different practice conditions in terms of perceived relevance. 

A second goal is to examine the effects of practice method on strategy knowledge and 
application of writing strategies. Several studies have observed a lack of difference 
between traditional practice and game-based practice for performance and achievement 
(e.g., Ke 2008; Klein and Freitag 1991). Thus, we expect that game-based practice may be 
as effective as conventional forms of practice in terms of learning achievement. However, 
game-based practice may not exceed other forms of practice in terms of achievement. 
Finally, we explore whether students’ perceived motivation differentially affects posttest 
achievement. 


Method 
Participants 


Participants (V = 175) were volunteer German university students enrolled in English 
courses to acquire and improve English fluency. The topic of the courses was English at 
level B2+ and Cl of the Common European Framework of Reference for Languages 
(Common European Framework of Reference for Languages: Learning, Teaching, 
Assessment (CEFR)). At level B2+ and C1 of the CEFR, a student is expected to use a 
wide range of language structures with few errors and express ideas and opinions on a 
variety of topics. They can exchange information reliably and have an active command of 
the essentials of the language. Students were assigned to the courses at the beginning of the 
semester via an electronic placement test. Instructional materials and practice were offered 
in English as a part of this curriculum. Participants were 28 % female with a mean age of 
22.54 years (SD = 2.27). Reported majors were primarily technical (37 % engineering, 
26 % computer science, 17 % economics, 14 % environmental science, and 6 % other). 


va Springer 


A. Proske et al. 


poseq 
-SUNLM 


486 


poseq 
-uonsond 


syuouryst[durosov pue syse} s Joureay [[e 1OJ 
Spiepur)s JUSWIOINSROU JUa}SISUOD oprAold pue ‘suOTEIDEdxe payeis YIIM JU}SIsUOD sjUaUTETINba1 souRUIOJIod OyRIAI 


SSodNS PIBMOAI 0} ( [J] pure 
MOYS,,) SWOJJO ITO} JO s}[Nsor ay} JUaserd syuopn)s 39] JO ‘SaATJUDOUT puR ‘spreMol dI[OquIAS JO Tear ‘asTeId [eqIOA IS) 


sura{qoid 
«PHOM-[B9I,, DATOS MOU ued Ady] MOY 9S 0} SJUSPN|S MOTTE Jey} sodures YOM Jo ‘suone[NUTIS ‘suaTqord aptAolg 


sgfdiourid uonorjsnes 
WoJJo [euossed 0} ssadons saynquye yey) Yorqpoey epraoid pue ‘foNUOS [euossed Jayjo yey) sonbruyse} os) 


ssooons 
SUIUILI] OSRIIOUT YOIYM sooudLIodxa Sulsusyfeyo pue ‘porea ‘Aueut SurprAoid Aq douajaduro0o ul Jorjoq aseosoUy 


BIIOIIO DATJENTLAD OY} PUL SsoddNs IO} syuoWoTINbal oy SuruTetdxa Aq suotyejoodxo saIsod pure jsny YsT[quisy 
satdiourid souspyuog 
YIOM ,SIBUIVI] BY) 0} Poye]or soisoyeue pue soydwiexa ajo1ouo0s SurprAoid Aq Jerry s}daouod puv s[eLisyeu ay) Dye 


sjepour for aantsod pur ‘sontiqisuodsar dryssapesy ‘samtanoe aateiadooo 
‘sontunjioddo juowoaaryor euosied Surpraoid Aq sanjea puke SdATIOW IJOUIvIT 0} DATSUOdSaI UOTONNSUI OyeIA 


WY) DUYop SIOUIes] OALY JO s[eoS JUosaid Joye pue ‘UOTONISUT oY} JO AITPNHN oy} Jo sojdurexa Jo syuoua}eIs OpIACIg 
so[droutid soueas[oyy 


syUoAo 
poayodxoun pue ‘saydurexa jsarajur ueumMy ‘sarZopeue ajJa1oU0d ‘a[A}s UOTRJUASOId UT SUOTBIIVA YIIM JsorO]UT UTeIsNS 


soSuayyeyo Suryury) SuLmmyinu pue ‘Armbur Sunesiouss ‘soxopesed Sunes ‘suonsenb Suryse Aq Aytsonmno aseaiouy 
[elioyeul [euoTOWD JO/pue [euosiad ZuNooful ‘soyovoidde Jaaou Suisn kq JUIWLIApUOM ‘AJISOLIND aeIID 


sgfdiound uonueny 


soIsaqens SOUV 


so[diourid uSIsap SOWV YUM sonovId Jo SUIO}J JWoIOIJIp Jo yuowUsITY T[ 9qUy 


pringer 


QS 


Game-based practice vs. traditional practice 487 


Their mean duration of enrollment was 5.51 semesters (SD = 2.80). Participants received 
a text on academic writing guidelines in compensation. 


Design and procedure 


A randomized, controlled experiment was conducted consisting of four phases: pretest, 
learning, practice, and posttest. Participants were randomly assigned to one of four 
practice conditions with a between-subjects design: game-based, question-based, model- 
based, and writing-based practice. There were no significant differences between the 
groups for gender, age, course of studies, semester, and English language skills. All 
materials were provided in English. 

After a pretest assessing prior strategy knowledge, all students studied strategies for 
writing an essay introduction by watching four short videos. These videos were obtained 
from the Introduction Building lesson of the computer-based writing strategy training 
system Writing Pal (W-Pal, McNamara et al. 2012; Roscoe and McNamara 2013; Roscoe 
et al. in press). W-Pal was developed to provide writing strategy instruction, practice, and 
feedback to high school students and entering college students. W-Pal synthesizes key 
principles of strategy instruction, modularity, extended practice, and formative feedback 
(Dai et al. 2011; Roscoe and McNamara 2013). The interdisciplinary development of the 
system spans over 5 years with input from cognitive psychology, linguistics, computer 
science, and English education. Several studies have demonstrated the effectiveness of 
W-Pal and guided iterative development and improvements to the system (e.g., modifi- 
cations of the user interface or mode of instruction, see for example Roscoe et al. 2013; 
Roscoe and McNamara 2013; Roscoe et al. in press). 

After viewing the instructional videos, students were offered different practice possi- 
bilities depending upon their assigned condition. Game-based practice (n = 41) consisted 
of playing Essay Launcher, a mini-game in which students applied some of the intro- 
duction building strategies to rescue stranded spaceships. Question-based practice 
(n = 46) was identical but all game elements (i.e., the narrative, graphics, and points) were 
removed. Game-elements were identified based on current game taxonomies (e.g., Bedwell 
et al. 2012; Garris et al. 2002; Wilson et al. 2009). Model-based practice (n = 47) 
prompted students to analyze examples of good introductions. The writing-based practice 
condition (n = 41) served as a control condition. Students in this condition were asked to 
practice the previously taught introduction building strategies by writing introductions on 
given prompts. All students were required to complete three rounds of practice. 

After practicing, students completed a posttest that elicited students’ motivational 
perceptions of the practice activity (IMMS, Keller 2010), and assessed both posttest 
strategy knowledge and application of introduction building strategies. The experimental 
session lasted about 90 min. 


Materials 

Learning materials 

The Introduction Building lesson consisted of four 5-min, strategy-focused videos (Roscoe 
et al. in press). The lesson began with an Overview video that presented the rationale for 
the lesson and previewed the strategies to be covered. Subsequently, three specific videos 


provided students with instruction on how to write introductions using the TAG mnemonic: 
Thesis Statements, Argument Preview, and Attention-Grabbing Techniques. Students 
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Writing an Effective Thesis Statement 


- every essay must have a thesis 


- must clearly state your position and 
directly answer the question in the prom 


- should incorporate a clear link or 7 
connection to your primary argumen 


Fig. 1 Screenshot of a video 


learned how to author each of these elements, and were provided with five techniques to 
make the introduction interesting (e.g., using a personal anecdote). Each video was pre- 
sented by an animated character and illustrations and examples appeared on a whiteboard 
(see Fig. 1). Completing the videos unlocked associated practice tasks, depending upon 
condition. In all practice conditions, information about the features of thesis statements and 
attention-grabbing techniques were available by clicking on a Help button. 


Conditions 


Game-based practice The mini-game was an identification practice task in which stu- 
dents examined given text excerpts to label key strategies covered in the lesson, or to 
identify how strategies may be used to improve the text (Roscoe and McNamara 2013). In 
this mini-game, the core game characteristics of fantasy, narrative, rules, goals, sensory 
stimuli, challenge, assessment, competition, and control were incorporated. Students 
applied several lesson strategies to rescue five stranded spaceships (fantasy, narrative), 
including choosing an appropriate thesis statement for the given paragraph and identifying 
the attention-grabbing technique exhibited in the paragraph (rules, goals, see Fig. 2). 
Incorrect answers triggered a failed launch animation with sound effects (sensory stimuli) 
and feedback on why the answer was incorrect. Correct answers rewarded the students with 
an animation showing the ship returning to Earth (sensory stimuli). Students had a total of 
ten attempts to save the five spaceships (challenge, control). At the end of the game, 
students earned points depending on the number of rescued spaceships (i.e. the number of 
correctly labeled paragraphs) and the number of remaining attempts (competition). Thus, 
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Prompt: Does modern technology support or hinder our creativity? 


Why do so many artists rely on computers to help them be creative? How does 
technology influence the creative process? Certain forms of music are impossible 
without special technology to create digital effects. Many filmmakers rely on 
computerized graphics to create fantasy or science-fiction scenes. Even authors use 


word processors to keep notes and speed up the writing process. 
Question 


HELP 


Thesis Repair Control 


) I prefer digital music and photography to old-fashioned music 
S and paintings. 


| ne 


Technology creates new tools and techniques, which expand 
the boundaries of creativity. 


EE 


Aska 1 > 
Question Technology destroys creativity because artists use computers 
too much. 


Fig. 2 Screenshot of game-based practice 


Prompt: Does life imitate art, or does art imitate life? 


Many people find role models in fictional characters in books and movies. Sometimes song lyrics can help a person see 
the world in a different way, and perhaps change their attitudes towards themselves or others. Although many people 
agree that art is important, they do not always agree on the role art plays in our lives. Some would argue that art is 
just an imitation of life. They would say that artists just recreate what they experience, but make it appear new. 
However, that view ignores the ways that art inspires us. 


hesis: 


@ Life often imitates art, especially when people are inspired to change their lives by a song or movie. 
© People who read more books and watch more movies are more creative, because they are inspired. 
@ There are many ways that life imitates art, such as when people dress up as of movie characters for Halloween. 


echnique: 


© Ask a Question 
@ History Review 
@ Set a Scene 

@ Anecdote 

@ Controversy 


Attempts remaining: 10 


Fig. 3 Screenshot of question-based practice 


students could judge the quality of their strategy application based on their game score 
(Roscoe and McNamara 2013). 


Question-based practice To equate practice materials across conditions, question-based 
practice was identical to the mini-game but all game elements were removed. As in the 
game, students were required to select appropriate thesis statements and identify attention- 
grabbing techniques (see Fig. 3). When students made errors, corrective feedback was 
delivered. In each round, students were allotted ten attempts to label five given paragraphs. 


Model-based practice Model-based practice consisted of studying five complete exam- 
ples of introduction paragraphs. These examples demonstrated all elements of introduction 
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Practice 1/3 
Prompt 1: Is it true that the more things change the more they stay the same? 
When was the last time you heard a new slang term? What happens when a biologist discovers a new species or a doctor encounters a new disease? In 


fact, new slang words become popular every few months, and scientific discoveries and new technology require new terms cribe them. Over time, 
cer s" of grammar can even fundamentally change. The ways that language evolves over time demonstrates that real change can occur, and 


that things do not stay the same. 


Click when you finished reading this Introduction. 


Fig. 4 Screenshot of model-based practice 


building strategies: effective theses, argument previews, and attention-grabbing tech- 
niques. Students were prompted to analyze these examples to learn how to reproduce the 
critical introduction elements later (see Fig. 4). 


Writing-based practice Students in the writing-based practice condition were prompted 
to use their knowledge of introduction building strategies to write introductions on three 
different prompts. In the instructions, students were told to practice formulating good 
thesis statements and different attention-grabbing techniques. 


Measures 
Instructional materials motivation survey (IMMS) 


The Instructional Materials Motivational Survey (IMMS, e.g., Keller 2010) was used to 
assess ARCS-related motivation. The IMMS is a 36-item measure that evaluates how a 
practice activity relates to each of the four components of the ARCS model (Rodgers and 
Withrow-Thorton 2005): Attention (12 items, Cronbach’s alpha = 0.82), Relevance (9 
items, Cronbach’s alpha = 0.75), Confidence (9 items, Cronbach’s alpha = 0.78), and 
Satisfaction (6 items, Cronbach’s alpha = 0.83). Survey items were adapted to the study 
by changing the word “this lesson” to “the practice activity” (e.g., “There was something 
interesting at the beginning of this lesson that got my attention.” was changed to “There 
was something interesting at the beginning of the practice activity that got my attention.”). 
Each item was rated on a five-point scale from “1” (not true) to “5” (very true). 


Strategy knowledge test 


Strategy knowledge was assessed twice, once before learning (pretest strategy knowledge) 
and again after practice (posttest strategy knowledge). The strategy knowledge test com- 
prised eight multiple-choice questions assessing students’ ability to (a) define and describe 
introduction building strategies, (b) recognize and apply introduction building strategy 
examples, and (c) understand the importance of introductions. Four questions had a single 
correct answer, and four questions had more than one correct answer option. Students 
received credit for choosing any correct answer. Strategy knowledge test scores are 
reported as a percentage of a maximum possible score of 19 points. 


Application of strategies test 


The strategies application test was administered after the posttest strategy knowledge test 
and consisted of ten multiple choice-questions in which students read example 
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introductions. Students completed the test by either (a) identifying missing thesis state- 
ments or attention-grabbing techniques, (b) labeling the thesis and attention-grabbing 
technique, or (c) improving the provided introduction. Application test scores are reported 
as a percentage of a maximum possible score of 20 points. 


Time on task 


The time students spent watching the videos and practicing was calculated from log-files 
recorded by the system. 


Statistical analyses 


The first analysis examined whether students perceived game-based practice as more moti- 
vating, according to the ARCS model, than more conventional methods of computer-based 
writing strategy practice. Preliminary analyses revealed significant differences between the 
four practice conditions for both video-viewing and practice time (see Table 2). Post-hoc 
comparisons using the Tukey HSD test indicated that students in the question-based practice 
condition invested significantly less time watching the videos than did students in the writing- 
based practice condition (p = 0.021). Furthermore, practice time in model-based practice 
was significantly shorter than in the other three conditions (p < 0.01 for comparison with all 
conditions). Consequently, Pearson’s correlation coefficients were computed between the 
IMMS scales, prior strategy knowledge, video-viewing time, and practice time to determine 
whether these variables needed to be included as covariates. The IMMS subscales did not 
significantly correlate with any of these variables. Thus, we conducted a MANOVA including 
the IMMS subscales as dependent variables and experimental condition as the between- 
subjects factor (game-based, question-based, model-based, writing-based practice). 

The second analysis addressed the effects of different practice conditions on achievement. 
Posttest achievement was measured using a strategy knowledge test and an application test. 
Separate analyses of covariance (ANCOVA) were conducted to assess the between-subjects 
effect of practice condition on the two posttest measures. Following (Rausch et al. 2003), 
prior strategy knowledge was used as a covariate for both analyses. There were also signif- 
icant, positive relationships between video-viewing time and posttest strategy knowledge 
(r = 0.18, p < 0.05). Thus, video-viewing time was entered as a further covariate for the 
strategy knowledge posttest. Preliminary analyses evaluating the homogeneity-of-regression 
slopes assumption indicated that the relationship between the covariates and the dependent 
variables did not differ significantly as a function of the independent variable. 

The third analysis explored whether students’ ARCS-related motivation differentially 
affected posttest achievement. Two separate hierarchical regression analyses with inter- 
actions were performed. The criterion variables were the two posttest measures, i.e. the 
strategy knowledge and application of strategies tests. Students’ (a) prior strategy 
knowledge, (b) perceived attention, relevance, confidence and satisfaction of practice, as 
well as (c) three dummy-coded variables representing information about the condition 
(with writing-based practice serving as the reference group) were entered at step | (Model 
1). The interaction between each dummy-coded variable and each IMMS scale was added 
at step 2 (Model 2).' All predictor variables were centered. Following Aiken and West 
(1991), we conducted simple slope tests to probe significant interactions. 


' Similar regression analyses were conducted additionally including video-viewing time and practice time 
as predictors. The pattern of results did not change when they were included. 
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Results 
Prior strategy knowledge 


Table 2 presents the means and standard deviations for students’ prior strategy knowledge. 
A one-way ANOVA showed no significant differences between the four groups with 
respect to prior strategy knowledge. 


ARCS-related motivation 


Table 2 and Fig. 5 show the descriptive statistics for the IMMS scales assessing students’ 
ARCS-related motivation as a function of condition. A MANOVA revealed a significant 
main effect of condition (Wilk’s A = 0.73, F(4,168) = 4.74, p < 0.01, partial nv = 0.10). 
This result indicates that the effect for group differences in the MANOVA accounted for 
10 % of the group-differences. This exceeds the cutoff for a medium effect size using the 
guidelines proposed by D’Amico et al. (2001) in which values of 0.01, 0.06, and 0.14 
indicate small, medium, and large effect sizes when measured by partial eta squared (see 
also Richardson 2011). Univariate follow-up analyses indicated that main effects were 
significant for the Attention and Confidence subscales and marginally significant for the 
Satisfaction subscale (see Table 2). The Satisfaction subscale accounted for 4 % of the 
group-differences, whereas both the Attention and Confidence subscales accounted for a 
somewhat larger proportion of 5 %. This exceeds the criterion for a small effect size 
(D’ Amico et al. 2001; Richardson 2011). This effect size is consistent with the literature 
that also found small effects of a game on motivational measures (e.g., Bai et al. 2012). 
Post-hoc comparisons using the Tukey HSD test indicated that the mean score for the 
Attention subscale was significantly higher for the game-based practice condition than for 
question-based practice (p = 0.047). Furthermore, perceived confidence in the question- 
based practice group was marginally higher than in the model-based practice group 


(p = 0.072). 


ARCS-related motivation 


Attention Relevance Confidence Satisfaction 
fi Game — Question [J Model [J Writing 


Fig. 5 Mean motivational attitudes towards practice method. Error bars represent standard deviations 
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Posttest strategy knowledge and application of strategies 


Table 2 presents the means, standard deviations, and indicators of statistical comparisons 
for the posttest measures of strategy knowledge and application of strategies (see also 
Fig. 6). As shown on the left side of Fig. 6, the ANCOVA on posttest strategy knowledge 
did not reveal a significant effect of condition. However, as displayed on the right side of 
Fig. 6, the ANCOVA on posttest performance in terms of application of strategies showed 
a significant effect of condition which accounted for 6 % of the between subjects variance. 
This exceeds the criterion for a medium effect size (D’Amico et al. 2001; Richardson 
2011). Follow-up tests were conducted to evaluate pairwise differences among the adjusted 
means for practice condition. The Bonferroni procedure was used to control for Type I 
error across the pairwise comparisons. The results showed that students in the model-based 
practice condition performed significantly better, controlling for the effect of their prior 
strategy knowledge, than did students in the writing-based practice condition (p = 0.011). 
In addition, practice by answering questions marginally outperformed practice by writing 
(p = 0.056). There were no statistically significant differences between game-based and 
conventional forms of practice. 


Role of perceived ARCS-related motivation for posttest achievement 


Two separate, two-step hierarchical regression models were tested to investigate whether 
students’ perceived motivation differentially affected posttest performance in the four 
conditions. Results of these analyses are presented in Table 3. For the posttest on strategy 
knowledge, Model 1 (including the main effects of prior knowledge, ARCS-related 
motivation, and condition) accounted for 14 % of the variation in posttest strategy 
knowledge. The results show that only prior strategy knowledge significantly predicted 
posttest achievement. Introducing the interaction between each dummy-coded variable and 
each IMMS scale (Model 2) explained an additional 10 % of variation in posttest strategy 
knowledge, but this change in R? was not significant (F(12,154) = 1.78, p = 0.06). 


Posttest achievement (%) 


Strategy knowledge Application of strategies 


fH) Game — Question §] Model! fi Writing 


Fig. 6 Achievement on the posttest measures of strategy knowledge and application of strategies. Error 
bars represent standard deviations 
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For posttest performance of application of strategies, Model 1 explained 15 % of the 
variation in posttest application of strategies. Adding the interaction terms as predictors 
explained an additional 15 % of the variation in posttest application of strategies and this 
change in R? was significant (F(12,154) = 2.67, p < 0.01). Three interactions were 
significant: Writing versus Game x Relevance, Writing versus Question x Satisfaction, 
and Writing versus Game x Satisfaction (see Table 3), suggesting that the effect of 
perceived ARCS-related motivation on posttest application of strategies depended on the 
condition. 

Simple slopes for these associations were tested for low (—1 SD below the mean) and 
high (+1 SD above the mean) scores on the respective IMMS scale. The effect of practice 
condition on students’ posttest strategy application performance depended on perceived 
relevance of practice. Students in the writing-based practice condition who perceived the 
practice activity as less relevant performed poorly in comparison to those who perceived 
high relevance of the writing-based practice (#1171) = 2.34, p < 0.05) and in comparison 
to those in the game-based practice condition (t(171) = 3.16, p < 0.01). By contrast, there 
was no effect of perceived relevance in the game-based condition. Figure 7 plots the 
simple slopes for this interaction. 

The effect of practice condition on students’ posttest strategy application perfor- 
mance further depended on perceived satisfaction of practice. Those in the writing- 
based practice condition who experienced more satisfaction performed poorly in 
comparison to those who reported more satisfaction with game-based practice 
(t(171) = 2.23, p < 0.05) and in comparison to those in the question-based practice 
(t(171) = 3.03, p < 0.01). Furthermore, those in the writing-based practice condition 
who reported more satisfaction performed poorly in comparison to those who expe- 
rienced less satisfaction to the writing-based practice in the writing-based practice 
condition (game-based practice: f(171) = 2.64, p<0.01, question-based practice: 
11171) = 2.58, p < 0.05). By contrast, there was no effect of perceived satisfaction in 
the game-based and question-based conditions. Figs. 8 and 9 plot the simple slopes 
for these interactions. 
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Fig. 7 Simple slopes for writing-based and game-based practice conditions predicting application of 
strategies at low and high relevance of practice 
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Fig. 8 Simple slopes for writing-based and game-based practice conditions predicting application of 
strategies posttest at low and high satisfaction of practice 
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Fig. 9 Simple slopes for writing-based and question-based practice conditions predicting application of 
strategies posttest at low and high satisfaction of practice 


Discussion 


Learning writing strategies requires a significant motivation to practice (Kellogg and 
Whiteford 2009). The aim of this study was to investigate (a) the degree to which game- 
based practice in writing strategy training was perceived as more motivating according to 
the ARCS model (Keller 1987a) than conventional methods of practice, (b) the degree to 
which game-based practice positively influenced students’ knowledge and application of 
writing strategies, and (c) the degree to which students’ perceptions of practice differen- 
tially affected posttest achievement in the different practice methods. Game-based practice 
was compared to a similar question-based practice condition without game elements, as 
well as two traditional forms of practice (i.e., model-based and writing-based practice). 


va Springer 


Game-based practice vs. traditional practice 499 


Effects of game-based practice on motivation 


Students perceived game-based practice as significantly more interesting and engaging 
than question-based practice. These two tasks were isomorphic; the practice tasks were 
identical except for the game or non-game context. Thus, game-based practice appears to 
have the potential to grab students’ attention. This is important because gaining and 
sustaining learners’ attention to the instructional content is considered crucial for learning 
(Keller 1987b). 

Although descriptive data point in the expected direction, students perceived game- 
based practice as equally motivating (according to the ARCS model) compared to model- 
based or writing-based practice. Practice by analyzing model texts or trying to apply an 
acquired writing strategy is common in university classrooms. For example, writing style 
guides and textbooks frequently provide deconstructed examples along with worksheet 
style practice opportunities (e.g., Hacker and Sommers 2011). Thus, students may have 
perceived these practice methods as being relevant to their prior learning experiences, 
current learning objectives, or future use of the new writing skills. Furthermore, both the 
modeling and writing practice conditions asked students to focus on the concurrent 
application of the new introduction building strategies either in analyzing the given models 
or in trying to use the strategies. In contrast, game-based practice provided a concentrated, 
short experience for only some of the new introduction building strategies. Thus, students 
in the model-based and writing-based practice conditions, more so than students of the 
game-based practice condition, may have experienced their practice conditions as a means 
to improve their personal writing (Schunk and Swartz 1993). For this reason, they may 
have judged their practice to be more attention-holding, confidence-building, and satis- 
fying than expected. In addition, participants of the game-based practice group were 
required to play the game and playing time was pre-defined. In leisure time, game-players 
typically decide themselves which game to play and how long to play it. It is possible that 
the lack of control on these decisions negatively affected students’ ARCS-related moti- 
vation towards the mini-game (e.g., Jakobsdéttir and Hooper 1995; Wouters et al. 2013). 

Although students who engaged in model-based practice performed better on posttest 
measures of strategy application, their expectations of success were lower than for those in 
question-based practice. Students’ confidence in model-based practice might have been 
diminished because studying high-quality models may have lead them to feel that 
achieving similar results would be beyond them (Zimmerman and Risemberg 1997). 
Confidence can be enhanced by providing feedback on the success of strategy application 
(Schunk and Swartz 1993). Thus, the results of this study suggest that feedback on stu- 
dents’ progress, as realized in the question-based and game-based practice conditions, is of 
particular importance when designing motivating practice tasks in computer-based writing 
strategy training (Cameron and Dwyer 2005; Ricci et al. 1996; Rieber 1996). 


Effects of game-based practice on achievement 


As expected, and in line with prior research, game-based practice was just as effective as 
conventional practice forms in terms of achievement (e.g., Ke 2008; Klein and Freitag 
1991). Although practice by answering questions slightly outperformed practice by writ- 
ing, game-based practice did not. Question-based practice was nearly identical to game- 
based practice in terms of the cognitive processes required during practice. Thus, game 
features may have affected practice effectiveness. For example, students often begin new 
games with a trial-and-error strategy before reading instructions, which could decrease 
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meaningful utilization of practice time (Dempsey et al. 2002). This interpretation is sup- 
ported by prior research in which short-term use of mini-games also was associated with 
lower achievement (Jackson et al. 2011). Other research has shown that students learning 
with a text aligned to ARCS design principles showed significantly lower knowledge 
acquisition in an immediate posttest (Astleitner and Lintner 2004). However, these neg- 
ative effects of game-based practice or alignment with ARCS design principles disap- 
peared when a long-term perspective over several weeks was considered (e.g., Astleitner 
and Lintner 2004; Jackson et al. 2011). 

The results here indicated that students in the model-based practice condition signifi- 
cantly outperformed writing-based practice in terms of strategy application at posttest. This 
advantage of model-based practice is in line with research showing that encouraging 
students’ reflection fosters learning (e.g., Garris et al. 2002; Knudson 1989; Moreno and 
Mayer 2005; Wouters et al. 2013). Furthermore, the results of this study indicate that 
asking students to apply a new strategy without specific instructional support or feedback 
may not be a very effective practice method in writing strategy training (Leemkuil and de 
Jong 2011). This approach may be more suitable in later stages of skill acquisition when 
students have gained more control over their strategy application via sufficient practice of 
the individual techniques (Kellogg and Whiteford 2009; Zimmerman and Kitsantas 1999). 


Differential effects of ARCS-related motivation on achievement 


Hierarchical regression analyses revealed that the relationship between perceived rele- 
vance and posttest strategy application, as well as satisfaction and posttest strategy 
application, was different in the game-based and writing-based practice conditions. 
Whereas both motivational components did not influence strategy application in game- 
based practice, they seemed to differentially affect strategy application in writing-based 
practice. These results suggest that game-based practice may particularly improve 
achievement for students who perceive conventional practice methods as irrelevant (Means 
et al. 1997). 

Unexpectedly, students reporting satisfaction with writing-based practice performed 
more poorly than students reporting low satisfaction with writing-based practice. This 
result requires additional research to provide a coherent explanation. Post hoc, one might 
speculate that those who invested less effort into the writing process were more satisfied 
because the task was less challenging—in turn, a consequence may have been lower 
performance. By contrast, those who invested greater effort into the writing task may have 
been more frustrated (e.g., by lack of feedback), but they also may have learned more. 
Unfortunately, these potential relations cannot be fully tested within the current study. 


Limitations 


One potential concern regards the stability of the IMMS. For example, several studies that 
have investigated the validity of the IMMS by confirmatory factor analysis found varying 
results with respect to the IMMS structure and item number (e.g., Huang et al. 2006; 
Loorbach et al. 2014). Notably, however, the IMMS is designed to measure situation- 
specific attitudes rather than more stable psychological traits (Keller 2010). Thus, varying 
outcomes when using this measure may be explicable by the situation-specificity of the 
IMMS (e.g., Huang et al. 2006). Along these lines, the objective in this study was to not to 
measure students’ generalized level of motivation toward one practice activity in 
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computer-based writing strategy training compared to another. Rather, our goal was to 
compare students’ self-reported reactions across different situations (i.e. different practice 
activities) guided by an established motivational design model. To this end, the IMMS’ 
sensitivity to situation-specific motivation was considered to be one of its strengths. For 
example, we found that students in all conditions reported a moderate to high levels on the 
Attention, Relevance, and Confidence sub-scales of the IMMS. However, they did not 
perceive high levels of satisfaction from practicing. These results indicate that there is 
room for motivational improvement in all forms of practice (e.g., Huang 2011). As such, 
the IMMS items can identify potentially critical motivational components of particular 
interest to instructional designers when designing practice forms in computer-based writing 
strategy (e.g., Huang et al. 2006). 

A second potential concern regards the generalizability of our results to other games. 
The Essay Launcher was the sole mini-game used within this study. This game was 
selected for methodological reasons from a collection of mini-games accompanying the 
introduction building lesson of W-Pal (McNamara et al. 2012; Roscoe et al. in press). This 
mini-game focuses on the learning objectives of identifying an adequate thesis statement 
and distinguishing different attention-grabbing techniques. The application strategies test 
also requires students to identify that a thesis statement is missing or to improve intro- 
ductions. These skills were not explicitly practiced by the game-based and question-based 
practice groups. Transfer of acquired knowledge is considered as a function of the simi- 
larity in cognitive processes engaged by the game and the application test (e.g., Tobias 
et al. 2011). Thus, future research may need to compare traditional methods of practice to a 
collection of multiple mini-games that more comprehensively cover all learning objectives 
of the strategy lessons (e.g., Ke 2008; Rosas et al. 2003). 

Third, practice time in this study was limited due to methodological reasons. However, 
given the assumption that motivating practice influences effort rather than performance 
(Huang 2011; Keller 1987b), it may be that the length of practice was too short to have a 
measurable effect (e.g., Wouters et al. 2013). Furthermore, it is possible that observed 
outcomes would not be sustained once the novelty of game-based practice wears off. This 
suggests that investigating the effectiveness of mini-games for practice should be con- 
ducted in a more longitudinal manner, for example by giving the opportunity to practice 
several times over several weeks (e.g., Jackson et al. 2011; Ke 2008). In addition, 
investigating motivational pretest—posttest gains from practicing would allow deeper 
insight into the complex interplay between motivation and achievement in computer-based 
writing strategy training (e.g., Wouters et al. 2013). 

A fourth concern regards the design of the conditions in this study. Our aim in this study 
was to compare traditional practice forms to game-based practice. Maintaining the eco- 
logical validity of writing-based and model-based practice necessarily led to differences in 
the instructional design of the conditions. To control for this, question-based practice was 
developed nearly identical to game-based practice. However, to better separate motiva- 
tional effects of game-based practice, future studies should use more strictly equivalent 
versions of conditions (e.g., Klein and Freitag 1991). This would allow more closely 
probing specific circumstances for effectively fostering practice motivation by means of 
ARCS design principles. In addition, the effects of different forms of practice may be 
guided by ARCS design principles in different ways. For example, the features that best 
support student attention and confidence in practice games may not be the same features 
that achieve these outcomes in more conventional practice. One goal for future research 
may be to identify features with the greatest impact within or across different practice 
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forms. For this, it is recommended to adapt the IMMS to the objectives of the particular 
evaluation (e.g., Huang et al. 2006). 

A final consideration is that the effects observed in this study might vary when 
investigating students working in their native language, or for native English speakers. 
This possibility certainly merits further study. 


Conclusion 


The results of this study underscore the necessity of interconnecting motivational and 
instructional design when developing practice methods for computer-based writing strat- 
egy training (e.g., Wouters et al. 2013). Game-based practice might increase the likelihood 
of student involvement with computer-based writing strategy training (i.e. devote extra 
time to practice), but it does not guarantee for better achievement. A sound instructional 
design which, for example, encourages students’ reflection on the application of the novel 
strategies and includes formative feedback, must also be apparent (Ricci et al. 1996). When 
designed in this manner, mini-games may be a valuable instructional tool and a promising 
method for facilitating students’ acquisition of writing strategies. It is also worth noting 
that the financial cost to develop multiple mini-games can be relatively low, making game- 
based practice both educational and cost-efficient. 
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